Search for: All records

The rapid growth of artificial intelligence (AI) and machine learning (ML) has led to significant innovations but also raised ethical concerns. Researchers and students designed an ethical online game in this study to spread awareness about making informed decisions when using AI and ML. Conducted within a directed research group (DRG) curricular method, the study engages students as co‐researchers to develop a game, from developing ideas to playtesting the game in a class setting. The study employs a quantitative methodology to analyze a survey that 32 students, each with diverse backgrounds and knowledge in game development, conducted after each class session over three semesters. Findings indicate that self‐reported engagement changes depending on the activities done in each session, with students feeling capable of contributing to research and game design. 

Historically, the African American female population has been underrepresented within the STEM workforce, creating non-inclusive environments. The metaphor of getting a seat at the table reflects the idea of inclusivity where Black women’s opinions are valued, and they can make decisions and create change within the workplace. Intersectionality poses several unique challenges for Black women pursuing careers in the video games industry. We motivate further research on this group through existing literature and insights from running a research group on the group design of human-centered data science games in a collaborative university setting. We take an autoethnographic perspective on the topic, with the first and second authors grounding their findings in their own experiences as Black women in STEM and higher education and with video game development studios. We outline tangible actions toward the recruitment, development, and retention of Black women in the video games industry in the future. 

From our smartphones to our social media, artificial intelligence (AI) algorithms are becoming ubiquitous in our everyday lives. However, the conveniences that they bring come alongside many potential social and political harms. It is imperative that members of the public develop data ethics literacy to interpret AI’s harms and benefits daily. The immersive and transformative nature of games may enable a wide range of people to explore complex ethical concepts in AI and data science through the lens of speculative design. In this project, we focus on the learning process of a diverse group of students from two universities as they embark upon a process of game design to teach ethical thinking in data science/AI. Through qualitative analysis of semi-structured interviews, we apply a speculative game design framework to identify aspects that aid student learning. 

Differential privacy (DP) data synthesizers are increasingly proposed to afford public release of sensitive information, offering theoretical guarantees for privacy (and, in some cases, utility), but limited empirical evidence of utility in practical settings. Utility is typically measured as the error on representative proxy tasks, such as descriptive statistics, multivariate correlations, the accuracy of trained classifiers, or performance over a query workload. The ability for these results to generalize to practitioners' experience has been questioned in a number of settings, including the U.S. Census. In this paper, we propose an evaluation methodology for synthetic data that avoids assumptions about the representativeness of proxy tasks, instead measuring the likelihood that published conclusions would change had the authors used synthetic data, a condition we call epistemic parity. Our methodology consists of reproducing empirical conclusions of peer-reviewed papers on real, publicly available data, then re-running these experiments a second time on DP synthetic data and comparing the results. 

Urban population growth has significantly complicated the management of mobility systems, demanding innovative tools for planning. Generative Crowd-Flow  (GCF) models, which leverage machine learning to simulate urban movement patterns, offer a promising solution but lack sufficient evaluation of their fairness–a critical factor for equitable urban planning. We present an approach to measure and benchmark the fairness of GCF  models by developing a first-of-its-kind set of fairness metrics specifically tailored for this purpose. Using observed flow data, we employ a stochastic biased sampling approach to generate multiple permutations of Origin-Destination  datasets, each demonstrating intentional bias. Our proposed framework allows for the comparison of multiple GCF  models to evaluate how models introduce bias in outputs. Preliminary results indicate a tradeoff between model accuracy and fairness, underscoring the need for careful consideration in the deployment of these technologies. To this end, this study bridges the gap between human mobility literature and fairness in machine learning, with potential to help urban planners and policymakers leverage GCF  models for more equitable urban infrastructure development. 

Differential privacy (DP) data synthesizers are increasingly proposed to afford public release of sensitive information, offering theoretical guarantees for privacy (and, in some cases, utility), but limited empirical evidence of utility in practical settings. Utility is typically measured as the error on representative proxy tasks, such as descriptive statistics, multivariate correlations, the accuracy of trained classifiers, or performance over a query workload. The ability for these results to generalize to practitioners' experience has been questioned in a number of settings, including the U.S. Census. In this paper, we propose an evaluation methodology for synthetic data that avoids assumptions about the representativeness of proxy tasks, instead measuring the likelihood that published conclusions would change had the authors used synthetic data, a condition we call epistemic parity. Our methodology consists of reproducing empirical conclusions of peer-reviewed papers on real, publicly available data, then re-running these experiments a second time on DP synthetic data and comparing the results. We instantiate our methodology over a benchmark of recent peer-reviewed papers that analyze public datasets in the ICPSR social science repository. We model quantitative claims computationally to automate the experimental workflow, and model qualitative claims by reproducing visualizations and comparing the results manually. We then generate DP synthetic datasets using multiple state-of-the-art mechanisms, and estimate the likelihood that these conclusions will hold. We find that, for reasonable privacy regimes, state-of-the-art DP synthesizers are able to achieve high epistemic parity for several papers in our benchmark. However, some papers, and particularly some specific findings, are difficult to reproduce for any of the synthesizers. Given these results, we advocate for a new class of mechanisms that can reorder the priorities for DP data synthesis: favor stronger guarantees for utility (as measured by epistemic parity) and offer privacy protection with a focus on application-specific threat models and risk-assessment.